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ABSTRACT 


This thesis is a study of the Aircraft Turbine Engine Component Improvement Program 
(CIP). The study examines some of the problems associated with determining benefits accrued 
from CIP. The major thrust of the thesis was developing a component selection methodology 
end an analysis pioccduce for aetectmy changes in logistics parameters. The data source was the 
Engine Component Information Feedback Report (ECIFR). Data for this report come from 
aviation organizational level maintenance activities and squadrons. The thesis reached the 
conclusion that the effects of CIP are more effectively assessed at the component level rather 
than at the system level. The thesis further demonstrated the logical and data collection 
difficulties encountered in the process of isolating and measuring the incremental benefits 


obtained from CIP expenditures. 


Ace@etsiva Yor | 
a. 6Q¥2am) NI 


“ HTt? Tea fe 


Wes tes ee cine cP abet Rican ae ctia dh os ld 
BO oss 
Ava@ilaviilty Coder 
avail aad/er | 
Dist | Speotal 


= s a iP-\ ! 
os Pte Moet aan 








TABLE OF CONTENTS 


EL. INTRODUCTION: 3.350.326 66. SS hte hee Kies PS DSRS AES | 
A: OBJECTIVES: 834.02 eG d Grea Vea Ua ba Re Re eae Be Re 1 
B. RESEARCH QUESTIONS ......... 00: cee cece eee eee 2 
C. THE AIRCRAFT COMPONENT IMPROVEMENT PROGRAM. .... 2 
Le “Goals. 25.2. eke CAMA ee ee ed ale, Aiea Re Bee 2 
2. Why a CIP For Aircraft Turbine Engines Should Be a Productive 
Expenditure 5.560559 fain eo hod Manes whee ge eR Ose 4 
3. Problems and Trends in CIP Funding ...........-.+-+---5 8 
D. SCOPE AND LIMITATIONS ..... 21. cece cee eee ees 13 
E. ASSUMPTIONS ° 3-4. c65-25% sae eevn eet ew ee hae 14 
Il. METHODOLOGY FOR SELECTING COMPONENTS ...........---- 15 
A. DESIGNING THE EXPERIMENT .........---. 0020525 e eee 15 
1. Ideal Environment .... 6.0.0.0. eee eee eens 16 
2. Real World Constraints ........ 0.0.00. 16 
B,. (MEFHOD USED oc non ci Ve Oe eh a eR ed eS A ee 19 
C. ASSUMPTIONS ABOUT THE METHODOLOGY ............. 23 


iv 





Ill. THEORETICAL FRAMEWORK FOR DETECTING CHANGES IN MTBF 


C. FORMULAE AND PARAMETER RELATIONSHIPS ........... 
D. MATHEMATICAL RELATIONSHIPS FOR A SIMPLE MODEL ... 
1. Failure Rate Analysis...... 2.2.0.0... 000 c cece eee 

2. Influences of Changes Ir Failure Rates .................. 

E. TECHNIQUES FOR DETECTING THE CHANGES IN MTBF. .... 
1. Analysis by Graphic Representation of Data............... 

a. Graphic Plots of Time Versus MTBF ................ 

b. Graphic Plots of Cumulative Engine Flight Hours Versus 


Cumulative Failures ........0.. 0.0.0.0 cece ees 


B:. -PRESENTATION: OF DATA® 5s): Goa) wtice open Rope, hes Shao ea 


C.- CONCLUSIONS isi oe cease One ise tess UE 4ae eee 


24 


29 








C.. (RECOMMENDATIONS - 20 2s 24 vapG tees Ye Pee Lees 49 


D. AREAS FOR FURTHER RESEARCH ..................0-... 52 
APPENDIX: tess od sek oot KAR P26 ESS SS + BUSSES Eee eae SS? 54 
BIBLIOGRAPHY. os sivucdb tee en ee EA eS Se DRS w Deeeee Ses 58 
INITIAL DISTRIBUTION 24.2.25.¢500%45.20 ee Se Bike 2a bese Reasons 61 


vi 








LIST OF FIGURES 


Figure 1.1 Navy and Total DoD CIP Budgets, FY70-FY90 ............... 10 
Figure 2.1 Generic CIP Milestone Timeline ........... 0.0.0.0 eee eee 18 
Figure 2.2 Component Selection Logic Diagram ................00005- 22 
Figure 3.1 Airborne Weapons System Schematic. ............-...--0005- 27 
Figure 3.2 Changes in MTBF During a Component’s Life-cycle. .......... 31 
Figure 3.3° Straight Line: ccs ay eek Ba A Le Ee ee ee SER HESS ae 
Figure 3.4 Increasing Curve ....... 0.0.0... ee 32 
Figure 3.5 Decreasing Curve .. 2.2.0... eee ens 33 
Figure 3.6 Kink downward .... 0.0.0... ee eee 33 
Bigure: 3.7 Kink Upward 1.0005. po seem eae Ree eke ss es © SES 34 
Figure 3.8 Observed Monthly MTBFs ............0.. 0.0 e eee eee eee 36 
Figure 3.9 Cumulative Failures Versus Cumulative Engine Hours.......... 37 
Figure 4.1 Afterburner Igniter Fuel Valve Observed Monthly MTBF ........ 42 


Figure 4.2 Afterbumer Igniter Fuel Valve Cumulative Failures Versus Cumulative 


Enigine ‘Hours; '2 i004 6 ok wh ace BREAST LG RR Ge E Powe 42 





LIST OF TABLES 


Table III.1 Observed Monthly MTBFs. .................000000 eee es 35 
Table 111.2 Cumulative Failures and Engine Flight Hours. ............... 35 
Table I1.3 Analysis of Variance for One-Way Classification ............. 38 
Table III.4 Analysis of Variance for the Data in Tables II.3 and 1.4. ...... 39 


Table IV. 1 Afterburner Igniter Fuel Valve ANOVA ..........-........ 44 


Vill 








I. INTRODUCTION 


A. OBJECTIVES 
The primary objectives of this thesis are: 
* to examine the problem of associating Component Improvement Program (CIP) 
costs with accrued benefits 
+ to explore the funding trends for CIP and determine if they are appropriate 


* to propose a methodology for observing and evaluating the effects of CIP on 
component logistics parameters 


* to demonstrate the problems of using existing databases for evaluating the effects 
of CIP on component logistics parameters 
The requirement to manage weapons systems on a life cycle basis grows more 
important as defense resources become more scarce. The ownership cost of an engine 
mav exceed the original acquisition cost by two fold or greater.’ This condition causes 
aircraft project managers to seek ways to reduce life cycle costs. Since CIP has more than 
just a safety objective, in its broader context, it maight also be a means cf Jignificantly 
reducing ownership cost. To ensure that an appropriate amount of resources are allocated 


to CIP, the cost/benefit relationship must be examined. 





' JR. Nelson, “Life-Cycle Analysis of Aircraft Turbine Engines: Executive Summary.’. The 
RAND Corp, Santa Monica. CA. March 1977, p. 27. 











B. RESEARCH QUESTIONS 
The most important question which the authors want to answer is: Is there a 
positive relationship between expending CIP dollars and improving logistics 
parameters. Specifically, is there an improvement which is statistically significant 
between a component population’s Mean Time Between Failure (MTBF) before and after 
an expenditure of CIP resources and fleet unplementation of an engine change proposal? 
To assist in answering this inquiry, the following questions were used to guide the 
research effort: 
* How does the budgeting process for CIP occur, that is, who are the key decision 
makers and what incentives do they face in allocating CIP resources? 
¢ How are the benefits of CIP presently measured? 
* What factors are influencing the existing levels of funding? 


« What quantitative methods are currently used to determine efficient allocations? 


C. THE AIRCRAFT COMPONENT IMPROVEMENT PROGRAM. 

In this section of the thesis, the objectives and functions of the Aircraft Engine 
Component Improvement Program (CIP) are presented. Logical and _ intuitive 
justifications for CIP are discussed. 

1. Goals 

In accordance with Navy policy, the Aircraft Engine Component Improvement 


Program has three objectives’: 


* NAVAIR INST 5200.35; ser AIR-536; dated 25 Jan 1982. 
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* to maintain an engine design which allows the maximum aircraft availability at the 
lowest total cost to the government (primarily production and support cost)’; 


* to correct, as rapidly as possible, any design inadequacy which adversely affects 
safety-of-flight; and 


* to correct any design inadequacy which causes unsatisfactory engine operation or 
adversely affects maintainability and logistic support in service. 


To meet these objectives, the Aircraft Engine Component Improvement Program 
performs four functions. The first three functions are stated explicitly in the CIP 
acquisition plan and the fourth is stated implicitly throughout the official documentation 


and literature.“ The four functions are: 


* Problem Solving. The rapid investigation and resolution of fleet proble ms (safety, 
readiness, or operability ). 


* Problem Avoidance. An aggressive program of mission testing, analytical sampling 
of fleet hardware and engineering analyses designed to forecast hardware wearout 
rates, life limits, and problems before they occur in the fleet. 


* Product Improvement. To develop and qualify design changes, repair procedures, 
and alternate sources (of parts and supplies throughout the life of an engine) where 
substantial savings in operation and support cost can be realized. 


¢ Product Maturation: Provide an infrastructure (support facilities and experienced 
engineers) to mature the engineering design for newly developed and fielded 
engines. 


* It is not possible to simultaneously maximize aircraft availability at the lowest total cost. 
The objective would be more correct if stated as: Given a fixed amount of dollars, maintain an 
engine design which allows maximum aircraft availability. 


“ Aircraft Engine Component Improvement Program, Acquisition Plan (AP) No. A42-48-0-50 
tev B, approved Aug 13, 1987. J. R. Nelson, et al. "Policy Options for the Aircraft Turbine 
Engine Component Improvement Program, Institute for Defense Analyses, Alexandria. WA, May 
1987, chapter II. 


2. Why a CIP For Aircraft Turbine Engines Should Be a Productive 
Expenditure 

The Aircraft Turbine Engine Component Improvement Program is a sustaining 
engineering effort. The requirement for such a program exists for several reasons. 

First, an aircraft turbine engine is an extremely complex mechanical device 
consisting of static and dynamic parts. Some moving parts are exposed to high speed 
rotational forces and other parts are subjected to rapid and repeated movement. Both 
dynamic and static components intemal to the engine are exposed to significant stress 
which induces unexpected failures. CIP provides a mechanism to re-engineer the 
component and reduce these failures. 

Second, both categories of parts are acted on by an extreme range of operating 
temperatures, ambient pressures, and ambient temperatures throughout an operating cycle. 
Such conditions contribute to rapid failure of poorly designed and engineered components 
and the eventual failure of even well designed and engineered components. In contrast 
to military engines, commercial aircraft engines encounter a reasonably stable and 
predictable operating environment. The operating envelope of a commercial engine can 
be characterized by start up, followed by a steady climb to cruising altitude, a gradual 
decent, landing, and engine shut down. Much of the operating cycle of a commercial 
engine can be viewed as "steady state,” i.e., constant rate of climb, constant engine 
thrust, constant speed at a constant altitude, and constant rate of decent. "Transient" 


points such as changes in power setting are short intervals and occur during: 





* start up 


e take off 
* transition to and from cruising altitude and speed 
¢ landing 


* engine shutdown 


Military engines, out of necessity, encounter continually changing operating conditions 
which are more hostile to engine components than an operational environment analogous 
to the commercial conditions. 

Third, it is only possible to effectively design an engine after the mission has 
been defined and the requirement for a new air platform exists. As a consequence of this 
restriction, aircraft turbine engines for military applications often enter the operational 
environment several years before they reach design maturity. Of the major systems which 
constitute a military aircraft, the powerplant historically requires the longest period of 
time to develop. The airframe, for example, can be engineered and adequately modeled 
using scale models and wind tunnels. Significant cost savings are realized during 
airframe development by using these techniques as opposed to requiring full scale 
operational engineering models. This is possible because the airframe is basically an air 
foil with a limited number of moving parts. Furthermore, the behavior of airframes 
moving through air is well understood and predicted accurately both in a mathematical 


sense and by engineering scale models. 





Even if it were possible to design aircraft turbine engines using scale models, 
the power plant development process is inherently longer in duration.” Studies have 
estimated that an airframe design reaches maturity in four to six years, and the propulsion 
system in six to eight years.° A function such as CIP is required because an engine 
typically leaves full scale development (FSD) before maturity. When the requirement to 
bring a new aircraft on line by a particular date is driven by operational factors and the 
engine can not remain in FSD an optimal length of time, CIP provides a means to 
complete the design maturation process after an engine is brought into operational service. 

Fourth, conscientious tradeoffs must occur in making the decision whether to 
bring an engine to maturity during Full Scale Development or after deployment via CIP. 
If, for example, one chooses to use FSD to bring an engine design to full maturity, the 
benefits of having a weapon system which meets all predetermined mission requirements 
must be weighed against the benefits of having the platform in the hands of the operators 
sooner. The longer the system remains 1n Full Scale Development, the more obsolete it 
will be when it becomes operational. The altemative is to shorten FSD and finish the 
maturation process with CIP. Given the opportunity, operators will frequently opt to use 
a weapon system that could, from an engineering view point, benefit from a longer FSD 
period. An operator would rather have a serviceable system that doesn’t quite meet all 


specifications than wait an additional period of time. Practically speaking, there isn't any 


* Interview with Dr. R. Shreeve, Naval Postgraduate School, Monterey, CA. Department of 
Aeronautical Engineering. 


“J.R. Nelson, et al, “Policy Options for the Aircraft Turbine Engine Component Improvement 
Program," Institute for Defense Analyses, Alexandria, WA, May 1987, p. II-3. 
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guarantee that the system will ever meet all specifications no matter how long full scale 
development continues. Furthermore, there are incentives for the program managers to 
field weapons systems on time and withing budget. Any attempt to keep an engine in 
FSD until maturity implies more time and money. Time and money are scarce resources 
to both program managers and operators. 

Because of present design process constraints, operational service entry time 
requirements, and engine complexities, many undiscovered and potential failure modes 
remain dormant until the engine is actually used in an operational environment. The 
Aircraft Turbine Engine Component Improvement Program provides a systematic way to 
develop fixes for design deficiencies which occur during operational use. 

Fifth, as a new weapon system such as an aircraft enters the operational 
inventory, it gives the users new capabilities. With these new capabilities, warfighters 
" increase their effectiveness as they develop new tactics and maneuvers. Operators receive 
encouragement to do this. The new tactics and maneuvers, however, put the aircraft in 
an operating environment which may expose the engine to stresses, cycles, temperatures, 
and pressures that the design team could not have possibly anticipated. The new 
operating environment brings weaknesses of the engine’s design to the surface. If the 
engine remained in the specified operating environment and design envelope, new 
weaknesses may not have surfaced. After the aircraft engine is fielded, FSD funds are 
no longer applicable. At the moment, CIP is the means to correct the weaknesses induced 
by new tactics. Thus, CIP provides a method of resolving engineering deficiencies that 


are revealed as a result of changes in the operating environment. 











Finally, CIP can improve reliability and durability through accelerated engine 


testing. The purpose of accelerated engine testing is to discover deficiencies in design 
before the fleet does. By compressing the engine’s life cycle in a simulated operational 
environment, a failure mode can be identified before it becomes a critical fleet problem. 
Unfortunately, not all key environmental factors, such as atmospheric pressure ratios, are 
fully simulated on the ground. 

In summary, both logical and intuitive justifications exist to support a 
continuing engineering effort for aircraft engines. The Aircraft Engine Component 


Improvement Program currently fills this need. 


3. Problems and Trends in CIP Funding 
The Component Improvement Program is a program element of the military 
services’ research, development, test, and evaluation (RDT&E) budgets. The service’s 
RDT&E budgets are broken down into six major programs. They are: Technology Base, 
Advanced Technical Development, Strategic Programs, Tactical Programs, Intelligence 
& Communications, and Defensewide Mission Support. CIP is a line item in the 
Tactical Programs budget account. 

In some instances, two or more of the uniformed services use the same engine or 
derivatives of the parent engine; for those engines, a lead service is designated to 
coordinate the establishment of priorities. Resource requirements are apportioned to 
individual services based projected usage of funds. 

During budget formulation each service develops its individual CIP budget 


request. After the service budgets are reviewed and revised through the Planning, 





Programming, and Budgeting System (PPBS), tentative funding levels are used to form 
a budget proposal. During the spring of each year, meetings are held for those engine 
programs which have joint users. Based on the figures determined via PPBS, the lead 
service, in conjunction with the follower services, determines which engineering efforts 
receive priority for both common and unique problems. From these priorities and with 
the dollars available, a listing is made stating what tasks will be performed. The results 
of this meeting are then given to AIR 536 who incorporates these tasking priorities into 
its budget proposal to OPNAV. 

If the Congressional appropriation is different from the proposed budget, 
further coordination with the lead service takes place. Common and unique problems are 
re-prioritized and funds are reallocated among the tasks to allow the best utilization of the 
funds. If the congressional budget is the same as the proposed budget, then the program 
is executed as previously planned. The lead service bills the follower services for any of 
the common or unique tasks which are accomplished. 

Each aircraft engine program engineer is responsible for putting together his 
or her budget and submitting it up the chain of command. The individual priorities are 
reviewed by a board in the NAVAIR organization which establishes a proposal for 
OPNAV. OP-514I prepares the final budget figures which are presented to OP-05. In 
OP-05, CIP competes against all other OP-05 RDT&E line items for funds. 

Since 1985 there has been a significant decline in Navy funding for CIP (see 
Figure 1.1). The budgets since 1985 only provided sufficient dollars to meet engineering 


deficiencies which were classified as safety of flight related. The remaining dollars were 








CONSTANT DOLLARS 1989 
CMILIlans3 





Figure 1.1 Navy and Total DoD CIP Budgets, FY70-FY90 
Source: NAVAIR 536B 


allocated to program overhead and maintaining the existing support facilities such as 
engine test cells. In essence, after meeting the safety of flight objectives and supporting 
the existing infrastructure, typically, there has not been any funding available to address 
other programmatic objectives. There have been few CIP efforts undertaken to 
specifically reduce life cycle cost. Some reductions in life cycle cost and benefits of the 
remaining goals have been realized, but only as a by-product of meeting a safety of flight 
need.’ 

There has been a constant level of funding for Navy CIP from 1988 through 


1990. If the FY91 budget request for Navy CIP is approved without any adjustments, the 


” Conclusions made by authors from interviews with NAVAIR and OPNAV personnel. 
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Navy will receive approximately $38.7M.° This would be a $2.69M increase in funding 
over 1990, ending a three year trend of zero growth in CIF. 

Funding for CIP is also affected by the Foreign Military Sales (FMS) 
program. A foreign user of aircraft engines purchased from the United States also 
contributes to the respective CIP engine program based on the number of engines the user 
nation owns. The baseline figure for computing the FMS contribution is determined by 
the number of dollars spent by DoD for a specific engine CiP program divided by the 
number of engines in the U.S. inventory. For example, if the United States contributes 
$13,500,000 for CIP for the F-XXX engine program and owns 400, the baseline figure 
per unit engine is $13,500,000/400 or $33,750. To determine an ally’s CIP contribution 
for the same engine, this figure is multiplied by the number of engines owned by the ally. 
If, for example, 60 engines are owned by various allied countries, their total contribution 
would be $2,025,000 (60 * $33,750), resulting in total funding for CIP of $15,525,000. 
The important relationship between FMS funding and CIP is that the more money the 
United States puts into a program, the more money an FMS user contributes. The money 
contributed to a CIP engine program by FMS is in addition to the CIP RDT&E funds 
appropriated by Congress. 

Why did DON experience a substantial decrease in CIP dollars between 1985 
and 1987? Four reasons which contributed to this problem were provided by OPNAV. 


These were: 


* 1989 constant year dollars 


il 





¢ overall tightening of the defense budget starting in 1986 
* previous CIP management and assessment practices within OPNAV 


¢ no effective method for briefing or educating senior managers about the long term 
benefits of CIP 


* an inability to quantify the marginal benefit associated with the marginal 
expenditure of CIP funds. 

The challenge facing Aircraft Engine Component Improvement Program 
managers in today’s constricting defense budget is determining efficient levels of 
resources that allow CIP to accomplish its objectives without degrading operational, 
materiai, and mission readiness. An even greater concer in these austere times is the 
impact of significant decreases in funds for CIP. 

Investigation of FY-90 congressional actions indicated only minor adjustments 
to the Navy CIP budget. For FY-90, the Navy requested $35.7M; it received $37.4M.” 
From this, the authors infer that if one wanted to increase the CIP budget, the increase 
must be requested and justified by the service during the planning and programming 
stages of PPBS. The drop in dollar resources for CIP experienced by the Navy between 
FY83 and FY90 is a problem caused by budgeting processes internal to the service. The 
problem is not apparently, the result of congressional cuts. The issue of CIP funding 
levels centers on the efficient allocation of RDT&E funds and the competition CIP faces 


from other RDT&E programs. 


* then year dollars 
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In summary, the Component Improvement Program provides one method 
whereby the goals of continuing or sustaining engineering for aircraft turbine engines may 
be realized. The problems which must be solved before CIP will receive funding for all 
program objectives are: 

* quantifying the marginal benefit of an expenditure of CIP funds 
* ensuring that as key decision makers for CIP are replaced, these new managers 
understand the goals and functions of CIP 
Without an effective solution to these problems, CIP becomes vulnerable to intemal 


budgetary cuts. 


D. SCOPE AND LIMITATIONS 

The engineering complexities of aircraft turbine engines forced the research effort 
. to focus on looking for benefits and changes at the component level rather than looking 
for benefits accrued by the engine at the system level. Een after an engine subsystem 
or component has been re-engineered, the direct benefits of this engineering effort may 
not be discernable at the system level. For example, as one group of subsystems or 
components is improving in reliability, there are other subsystems or Components whose 
reliabilities may be declining, creating a cancellation effect at the system level. 

The ideal component for the purposes of this study possesses the following 


attributes: 


* CIP resources were expended to provide a fix to an engineering deficiency 


* the end product of the CIP effort resulted in an engine change proposal (ECP) 








APN funds were obligated to implement the ECP fleetwide 


the ECP has been in the fleet for a minimum of two years so that the benefits of 
the fix can accrue and the analysis can be based on actual experience rather than 
an engineer’s best estimate 


data about the component are available to quantify the benefits. 


ASSUMPTIONS 

This thesis is wmitten for: 

individuals who possess a working knowledge of the DoD and DoN resource 
allocation process, 


individuals who have an understanding of basic statistics, logistics engineering, and 
management theory. 


program item managers who prepare RDT&E budgets for upper echelon staff and 
Congressional review. 
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Il. METHODOLOGY FOR SELECTING COMPONENTS 
This Chapter explains the procedures and techniques used to identify data which 
would help determine whether there is a statistically significant difference between a 
component population’s Mean Time Between Failure (MTBF) before and after an 
expenditure of CIP resources and fleet implementation of an engine change proposal. In 
other words, can a causal relationship between CIP and improved performance as 


measured by MTBF be demonstrated? 


A. DESIGNING THE EXPERIMENT 
Given the freedom to evaluate the effect of CIP funding on logistics parameters, 
how would one organize such an experiment? 
First, a procedure which measures effectiveness must be formulated. In the context 
of this thesis, CIP effectiveness is defined as improvement in logistics parameters. The 
measure of effectiveness that will be used for the remainder of this thesis is increased 
MTBF because operating and support costs are largely affected by a component's 
MTBF”. 
Although CIP affects the overall readiness of an aircraft engine, it is difficult to 
assess discrete results as they affect total system performance. This 1s because as one 
'° Although this thesis uses MTBF, there are other logistics parameters which might be used 
as a measure of effectiveness for CIP. These are: maintenance dollars expended per component 
maintenance action, maintenance man hours per operating hour, in-flight aborts per 1000 flight 
hours attributable to this component, ground aborts per 1000 flight hours attributable to this 


comporent, Class ‘A’ mishaps per 1000 flight hours attributable to this component, and reduced 
number of aircraft required because of increased availability. 
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component's performance is deteriorating, another’s may be improving, thereby offsetting 
the declining performance. Conversely, as one component’s perfcrmance is improving 
as a result of CIP, the benefits may be offset by another component whose performance 
is deteriorating. Because of this intricate balance, the benefits of CIP must necessarily 
be assessed at a component level. The changes in a component’s performance are not 
usually affected by changes in the performance of other components, except in the case 


of catastrophic or induced failures. 


1. Ideal Environment 


In an ideal environment one might use the following procedure: 


¢ Test the hypotheses: 


H,: CIP has no measurable effect on a component MTBF. 
H,: CIP has a measurable effect on a component MTBF. 


* Select a random sample of aircraft engine components of a statistically significant 
size for evaluation. 


* Select those components that have been subjected to CIP funded re-engineering 
efforts. 


¢ Compare changes in component MTBF at periodic intervals. 
¢ Analyze the data and determine if the null hypothesis should be accepted or 


rejected. In other words, determine whether the MTBF changed significantly after 
subjecting a part to CIP. 


2. Real World Constraints 
Will the proposed methodology adequately assess the effectiveness of CIP 


funding in conjunction with real world constraints? What are the problems or 
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inadequacies of this methodology? Does the operating environment restrict or bias the 
data? 

Even after separating components into two sub-groups, one which has 
experienced CIP expenditures and the uther which nas not, this action may not truly 
isolate the effect of CIP funds for several reasons. 

First, even though CIP dollars were spent to improve the characteristics of a 
component, the effort may prove fruitless because of circumstances beyond the control 
of the CIP Program Manager. For example, CIP may develop an engineering change 
proposal, but because CIP covers only the research and development part of the process, 
if APN funds are not released to implement the "fix", the intended benefits are not 
realized. In essence, the CIP funds become a sunk and unrecoverable cost. 

The managers of APN accounts have several alternatives when it comes to 
allocating dollars. In the most general terms, they may designate funds to improve 
aircraft performance, increase aircraft warfighting capability, improve durability, or 
increase reliability. The first and second areas concem upgrading aircraft performance 
and the third and fourth areas concern reducing life cycle costs via improved logistics 
measures. There are tradeoffs between increasing aircraft performance or capability and 
increasing reliability and durability. Better reliability and durability may be foregone in 
an effort to achieve near term improved warfighting capability. In this case, the implied 
utility of CIP to APN managers is near zero. 


Second, when chere is a large investment in spare parts. there is a reluctance 


to dispose of these spare parts. Except in the case where CIP funded engineering changes 











involve safety issues, disposing of spare parts can be viewed as wasteful rather than as 
an irrelevant sunk cost. When the investment in spare parts is large and the 
improvements are in the areas of maintainability and reliability, there is a tendency to use 
the existing stock of old spares until it is =xhausted. This practice extends the transition 
period of going from the old component to the new version. The time to realize benefits 
is spread out over a longer period making it more difficult to isolate the effects of the re- 


engineered component. 
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Figure 2.1 Generic CIP Milestone Timeline 


Third. the steps of the CIP process do not occur instantaneously; time is 
required for research and develipment. Likewise, the benefits do not appear 
instantaneously; rather, they accrue over time. The generic life of a component, relative 
to CIP, is shown in figure 2.1. T, is the point wien a problem is discovered. T, is when 
CIP begins to research and design a solution for the problem. At T, the R&D is 


complete. The implementation of the fix starts at T, and it is fully fielded at T.. 








We assume that there is no observable change in MTBF prior to T,. MTBF 
is measured both at T, and again after T,. For the purposes of this study, components 
were chosen which were believed to have data available for all points on the time line. 
Discussions with individual engine program managers indicate that it takes a minimum 
of two years to go from T, to T,. In non-emergency situations it may take ten years or 
longer. The variable length of time from T, to T, makes it difficult to predict when the 
effects of CIP will manifest themselves. 

The above constraints indicate that one must consider more than just two 


categories of components relative to CIP. At least four categories are identified: 


* Components for which no CIP funds have been expended for redesign 

¢ Components which have been re-engineered with CIP funding but because of 
insufficient APN funding, they have not experienced the effects of the CIP 
expenditures 


* Components which have been re-engineered with CIP funding and are still in a 
transition period or awaiting fleet wide implementation of the solution 


¢ Components which have been re-engineered with CIP funding and have completed 
the time-line from problem identification to full implementation of the solution. 

Even with a variety of categories in which to place a component, it is often 

difficult to determine the correct one. Sometimes more than one category is correct for 


a component if several engineering efforts overlap. 
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B. METHOD USED 
To overcome the difficulties associated with identifying and categorizing 
components which have been re-engineered or "improved", components were selected 
which met the following criteria: 
¢ The component has exhibited a significant reduction in its failure rate (MTBF has 
increased). 
¢ There was an expenditure of CIP funds for Engineering, Research and Development 


* The component has gone through the CIP milestone timeline from problem 
identification through complete installation 


* Sufficient time has elapsed since the ECP was completely fielded (T,) to detect a 
change and observe a trend in the component MTBF 
A flow diagram of the logic used to identify candidate components is presented in Figure 
2.3 
The data used in this study spanned the years from 1979 through 1989.'' The 
purpose of the data search was to identify components that were "maintenance drivers” 
and had been subject to CIP. A component was considered to be a "maintenance driver" 
if it had a low MTBF, relative to other components of the same engine. “Maintenance 


drivers" were identified from the 1979-1982 historical data records. The list of 


'' The data for this thesis come from a document known as the Engine Component 
Information Feedback Report (ECIFR). The ECIFR is a database compiled from aviation 
maintenance (3-M) data and is used to monitor logistics parameters for aircraft engines. 
The data used to generate the ECIFR are maintained by NAVAIR. The data found in the 
ECIFR database were believed to contain sufficient detail to facilitate analysis of aircraft 
engine logistics parameters at the component level. 
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"maintenance drivers" was compared to 1989 ECIFR data to determine if the component 
was still a "maintenance driver”. If the component remained a "maintenance driver" it 
was not considered. If a component was no longer a "maintenance driver” it was 


considered for further analysis. 
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Is the component a 
"Maintenance Driver" in 
historical ECIFR? 


Does the component remain 
a “Maintenance Driver" in Yes 
the 1989 ECIFR? 





No 


Were CIP Funds spent 
on the component? No 


Yes 


Has the engineering change 


been implemented through- No 
out the fleet? 





Yes 


Has sufficient time 
elapsed since T5 No 


Yes 


CANDIDATE COMPONENT 


Figure 2.2 Component Selection Logic Diagram 











Through discussions with the engine program managers both at NAVAIR and at the 
contractor’s facility, the authors were able to determine if the component had benefitted 
from the expenditure of CIP funding. If the component was not a beneficiary of CIP 
funding, it was not added to the list of candidate components. After a list of candidate 
components was compiled, the year and month in which T, and T, occurred were 
determined with the help of the engine program managers. At this point an analysis of 


component MTBF would be conducted using the procedures found in Chapter II. 


C. «SSUMPTIONS ABOUT THE METHODOLOGY 
This methodology makes the following assumptions: 
¢ Any favorable growth in MTBF is a direct result of CIP expenditures and not the 
result of a more favorable operating environment. The severity of operational 


demands on the engine is assumed to remain constant. 


¢ Several years may lapse from the time a problem is identified until the solution is 
fully fielded. 


¢ Because of the rationale used to select candidate components, any negative impact 
CIP expenditures may have had on any MTBF are ignored. 


* The methodology considers only benefits which are measurable and quantifiable; 
it ignores any intangible benefits accruing to operators or any increased "utility." 


¢ In the absence of CIP, MTBF was in steady state and would have remained 
constant. 


* The data have built-in biases such as the amount of input error made by the 
maintenance data processors. This bias and others are assumed to remain constant 


over time. 


¢ All failures are independent; there are no induced failures. 
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WM. THEORETICAL FRAMEWORK FOR DETECTING CHANGES IN MTBF 

This Chapter discusses the theory which supports the following conclusion: changes 
in MTBF are more easily detected at the component level rather than at the system level. 
The terms and parameters used in the discussion are explained to enhance further 
understanding of the research problems. Included in this explanation are the mathematical 
relationships among the parameters addressed. Finally, an example using a hypothetical 
system is provided to illustrate the principles of the model. 

As the reader will note from the example that follows, large improvements in 
component parameters generate relatively small improvements in system parameters. 
Therefore, when attempting to determine the benefits of CIP, it is critical to evaluate them 
on the component level rather than on a system level. Otherwise, any improvement in 


a logistics parameter becomes lost in the "noise" of the system’s complexity. 


A. FRAMEWORK DEVELOPMENT 

A weapons system is an assemblage of integrated subsystems. For the purposes of 
this thesis, the system is an aircraft. An aircraft is the integration of four major 
subsystems: the airframe, the powerplant, the electronics suite, and the payload. Each 


subsystem is a combination of component weapons replaceable assemblies (WRAs) and 


sub-replaceable assemblies (SRAs). 





The attributes of the system are an aggregation of the attributes of its subsystems. 
Similarly, the attributes of a subsystem are an aggregation of the attributes of its 
component parts. 

Attributes of a system, subsystem, or component are initially estimated during the 
design phase before field data become available. Through user input field data are 
collected to evaluate reliability and other logistics parameters. After field data are 
accumulated, a comparison of the parameters from each set is made to measure the 
difference between design data and field data. If field data are determined to be more 
accurate, they should be used to revise projected life cycle costs and expected 


performance attributes. 


B. PARAMETERS AND DEFINITIONS 
When defining logistics functions, support costs, reliability and maintainability 
attributes, a component can be characterized by the following parameters: 


« MTBM. Mean time between maintenance. It is a function of both MTBF 
(MTBM,) and MTBM,. Equal to 1f(LA+Xfpt). Expressed in terms of time per 
maintenance action. 


« MTBF (MTBM.). Mean time between failures or mean time between corrective 
maintenance actions. ‘fhe average time between failures for a component, sub- 
system, or system. Usually expressed as operating hours per failure, i.e. 1000 
hours/failure. 


¢ 2 or failure rate. The rate at which units fail, usually expressed as failures per 
operating hour, i.e. .001 failuresfhour. 4 is equal to 1/MTBF. 


* MTBM,. Mean time between preventative maintenance. The average time between 


preventative maintenance actions. usually expressed as hours between maintenance 
actions, i.e. 100Qhrs per preventative maintenance action. 
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fpt. frequency of preventative maintenance. The rate at which units require 
preventative maintenance, expressed in units of preventative maintenance action per 


time. fpt is equal to 1/MTBM, 


Mct. Mean corrective maintenance time. The mean active maintenance time 
required to restore a system, subsystem, or component to an operational status after 
a failure has occurred. It is expressed in terms of time per corrective maintenance 
action. 


Mpt. Mean preventative maintenance time. The mean active maintenance time 
required to perform preventative maintenance. It is expressed in terms of time per 
preventative maintenance action. 

Mean active maintenance time. The wrench turning time required to perform 
maintenance, includes both Mpt and Mct. It is computed as a frequency weighted 
average. M=[Z(A*Mct)+2(fpt*Mpt)]/Z(A+fpt). 


MMH/OH Mean maintenance hours per operating hour. The average direct 
maintenance man hours per operating hour. 


MM$/OH Mean maintenance cost per operating hour. 


Reliability is the probability that a system or product will perform in a satisfactory 
manner for a given period of time when used under specified operating conditions. 


FORMULAE AND PARAMETER RELATIONSHIPS 


Neystem=2y 


MTBF=1/A 


A=1/MTBF 


These relationships assume that failures are random and independent from one component 


to another.’ 


". Complete derivations of the above formulae can be found in Blanchard, B. S., 


Logistics Engineering and Management, Prentice-Hall, Englewood Cliffs, NJ, 1986. 


26 








D. MATHEMATICAL RELATIONSHIPS FOR A SIMPLE MODEL 

To explain how a change in the failure rate, X, of one component might affect an 
entire system, the following hypothetical four subsystem aircraft is given as an example. 
This example illustrates the concept that large changes in component parameters do not 
necessarily translate into large changes in system parameters; in fact, the opposite is true. 
Large changes in component parameters result in relatively small changes at the system 


level. The weapons system in Figure 3.1 integrates the four major aircraft sub-systems 


in a series configuration. 





Figure 3.1 Airborne Weapons System Schematic. 


1. Failure Rate Analysis 

In the following example, each sub-system consists of 100 components in 
series. Such an assumption is an over simplification of an actual aircraft which consists 
of thousands of components both in series and in parallel. Components configured in 
parallel are usually identical in structure or provide a backup capability to another 
component. For example, on a multi-engine aircraft the engines are in parallel with each 
other, but as a whole, the propulsion system is in series with the payload. The failure rate 
of the aggregate propulsion system can be expressed as a single parameter. The 


assumption of a series system is made to reduce the mathematical complexity and 
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demonstrate that large changes in A at the component level yield substantially smaller 
changes in the failure rate at the system level. 

The failure rate, A, for each component will be assumed to be equal to the 
failure rate for every hts component, ie., A, ,=A,,=A,3=..-=A4 10; for each subsystem 


Agubaystem =LA=100A; for the whole system A, 


LA jcaubsystem=4* 100A=400A. Given the 


ystem— 


relationships above (p. 26), MTBF =1/(100A,)=0.01MTBF,, and 


sub-system 


MTBF, em=1/(400A,)=0.00025*MTBF,. Assume CIP resources are expended. This action 
changes the failure rate of only one component and leaves all other failure rates constant. 
If the failure rate of the one component improves by a factor of 2, ie. doubling the 


MTBF, that is A’=.5A then A 


‘subsystem 


=99.5A and i 


‘system 


=399.5A. 

For the above example and using the assumptions, decreasing the failure rate 
for a single component by 1/2 results in only a 0.5 percent improvement at the subsystem 
level and only 0.125 percent improvement at the system level. If it was possible to 


reduce the failure rate of a single component infinitely, ie. A’=0, then A, =99 and 


‘sub-system 


Asystem=399A resulting in a 1.0 percent improvement at the sub-system level and a 0.25 
percent improvement at the system level. 

Using the same example, as above, with four major sub-systems, let each sub- 
system consist of 100 components; assume all A’s are equal but one. Let one component 


have a failure rate equal to one hundred times that of the other ninety-nine components, 


A'=100A: then, Ags. sysem a=199A, Arup system s=100A, 2, 


sub-system c 


=I00A. X, 


‘sub-system d 


=100A. When 
resources are expended and A’ is decreased by a factor of two, then A” becomes .5)' 


which is the same as 502... This change can be observed at the sub-system and system 
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levels. Aju sytem Becomes 149A which is a 25 percent change in the subsystem failure rate 
and A,\tem becomes 449) which is a 10 percent change in the system failure rate. 

In the extreme case, assume ’ decreases infinitely, i.e., A"=0. When this 
change is reflected in sub-system and syste parameters Ay, syeem= 99%. Which is a 49.7 
percent change in the failure rate occurs and A,jem=399A, which is a 20 percent change 


in the failure rate. 


2. Influences of Changes In Failure Rates 

It is apparent that even large reductions in failure rates at the component level 
may only translate into very small changes in overall system parameters. Changes in 
failure rate (A) may be very difficult to detect at the system level and more easily 
observed at the component level. Even if a component is the driving force behind a sub- 
system failure rate, the effect of changes at the component level may still remain 
undetected at the system level. 

In summary, the authors conclude that the effects of CIP are likely to be 


undetected at the system level and should be appraised at the component level. 


E. TECHNIQUES FOR DETECTING THE CHANGES IN MTBF. 


To evaluate the effect of CIP efforts, the following assumptions are made: 


¢ This effect will trickle down, up, and through the sub-system and system levels. 
A change in the failure rate of a component will result in a change in the failure 
rate at the sub-system and system level. It has been shown that the changes at the 
sub-system and system levels are so small they may be undetectable. 


¢ Based on previous discussion, the effect cannot be detected at the system level; it 
is lost in the random noise associated with the operating characteristics of a sub- 


system or system. It can only be detected at the component level. 


29 














To evaluate the expenditure of CIP funds, a candidate component must be selected. 
The characteristics of the candidate component are then examined to see if there is a 
Statistically significant change in one or more characteristics. The data available from the 
ECIFR make it possible to evaluate failure rate, MTBF, MTBR, and/or MMH/OH. 
MTBF is analyzed by the model and techniques of this thesis because operating and 


support costs are largely affected by a component’s MTBF. 
1. Analysis by Graphic Representation of Data 


a. Graphic Plots of Time Versus MTBF 

One method of detecting a change in the MTBF would be to plot time 
in months against observed monthly MTBF. There are three general trends which one 
might observe when using this technique. MTBF could either be decreasing, remaining 
constant, or improving over time. A component could exhibit any or all combinations of 
the above during its life time. Figure 3.2 illustrates what a data plot might look like for 
a component during its life-cycle. The MTBF is relatively constant from months 1-i5. 
During months 13-17, there is a transition to a lower MTBF. It becomes relatively stable 
again between months 17-28. At month 28 there is a transition to a higher MTBF. At 


month 31 the MTBF appears to remain relatively stable for the remainder of the data. 


b. Graphic Plots of Cumulative Engine Flight Hours Versus Cumulative 
Failures 
A second method of identifying a change in MTBF would be to detect 


a change in the failure rate, A. Given the relationships A=|/MTBF and MTBF=1/(, it 
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Figure 3.2 Changes in MTBF During a Component’s Life-cycle. 


follows that a change in the MTBF will be reflected as an inverse change in the failure 
rate. Specifically, as the MTBF increases, A decreases, as MTBF decreases, A increases. 

If the cumulative number of failures ts plotted against operating time, the 
curve described by the data points can provide insight into the history of the failure rate. 
The slope of the curve (df/dt) at any point is the failure rate, A, at a given time. There 
are five general shapes of curves that data could possibly trace. These are shown in 


Figures 3.3 through 3.7. 





CUMULATIVE FAILURES 


TIME 'N CUMULATIVE BFH 


Figure 3.3 Straight Line 


A line with a constant slope indicates a component with a constant failure rate and 


MTBF. 


CUMULATIVE FAILURES 








TIME IN CUMULATIVE EFH 
Figure 3.4 Increasing Curve 
This curve shows a component with a continuously increasing failure rate or a 


continuously decreasing MTBF. 
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QMZal VE FAILURES 
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Figure 3.5 Decreasing Curve 


This curve indicates a component with a continuously decreasing failure rate or a 


continuously increasing MTBF. 


CUMULATIVE FAILURES 


TIME |N CUMULATIVE BFH 


Figure 3.6 Kink downward 


This curve shows a component with a constant failure rate until the bend in the curve. 


Beyond the kink the failure rate decreases to a lower constant value. 








OMULATIVE FA{LURES 


TIME (N CUMULATIVE EFH 


Figure 3.7 Kink Upward 


This curve again shows a component with a constant failure rate until the bend in the 
curve. In this case the failure rate increases (beyond the kink) to a higher constant 
value. 

Over time, an actual component’s failure rate may show any combination 
of the above graphs depending on where the component is in its life cycle and its 
continuing engineering efforts. These graphs provide a tool which help determine the 
trend a particular component is following relative to MTBF and failure rate 2. 

Example: Using the data from Tables II.1 and III.2, two graphs have 
been constructed. For the first graph, time in months versus MTBF is plotted; for the 
second graph, cumulative engine flight hours is shown versus cumulative failures. The 
data in Tables III.1 and Il.2 were generated for illustration purposes only and have no 


relationship to any actual engine components. 
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Table IlI.1 Observed Monthly MTBFs. 


cumulative cumulative cumulative 
failures flight hours failures 


343 
710 
995 
1232 
1539 


1775 
2096 
2433 
2732 
3065 
3527 
2899 


Table I11.2 Cumulative Failures and Engine Flight Hours. 
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cumulative 
flight hours 


4282 
4588 
4927 
5163 
5430 
5710 
5915 
6168 
6456 
6717 
7040 
7389 
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Figure 3.8 Observed Monthly MTBFs 


From Figure 3.8, one could conclude that at approximately month 13 
there was a significant increase in the component MTBF. After month 12, the MTBF 
remained relatively constant over time. In this example, CIP did in fact result in an 


increase in the component MTBF. 





EXAMPLE 11 


CUMULATIVE FAILURES 


343 1539 2732 4282 $430 6456 
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Figure 3.9 Cumulative Failures Versus Cumulative Engine Hours 


Figure 3.9 most resembles a kink downward plot which means that the 
_ failure rate was constant until approximately 4,282 cumulative engine hours. After 4,282 
cumulative hours one could conclude that CIP decreased the failure rate (A) to a lower 


constant value. 


2. Analysis by ANOVA (Analysis of Variance) 

A function of CIP is to reduce the life cycle cost of components. One could 
compare a population of component MTBFs before and after CIP engineering because life 
cycle cost is inversely related to MTBF. If there was a change in the population’s 
MTBF, and it was due to CIP, then it would be possible to quantify the change in terms 
of dollar savings. 

One technique for detecting a change in the mean of a population of 


components is ANOVA. In general terms at the component level, a change in a 
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parameter’s value may be detected by testing the hypothesis that the central tendency of 
the sample populations is the same, or the central tendency of the sample populations is 
different. In more common mathematical notation, the above statement is written as: 
Hy: by=}2=Hs--- 
H,: not all pt, are the same. 
The computations for an analysis of variance problem are usually summarized in tabular 


form as shown in Table 10.3.” 


Table 111.3 Analysis of Variance for One-Way Classification 


Source of Degrees of Sum of Mean Computed 
Variation Freedom Squares Square 


Factors SSA 


Error SSE 


Total SST 





Example: Using the data from Table III.1, test the hypothesis 1, = pL, at the 
0.05 level of significance. [i.e., that there is only a probability of .05 that p=]. 
MTBFs for months 1-12 make up the first sample population; MTBFs for months 13-24 


make up the second. 


° For a more rigorous explanation of ANOVA consult R. E. Walpole and R. H. 


Myers, Probability and Statistics for Engineers and Scientists, Macmillan Publishing 
Company, 1985. 
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Table [11.4 Analysis of Variance for the Data in Tables 01.3 and 0.4. 


SOURCE ss 
FACTOR 704.17 
ERROR 91.83 
TOTAL 796.00 


LEVEL N MEAN 
MTBF a 12 4.083 
MTBFb 12 14.917 


POOLED STDEV = 2.043 


INDIVIDUAL 95 PCT CI’S FOR MEAN 
BASED ON POOLED STDEV 





The results of the ANOVA are found in Table I1I.4.'° Before any meaningful 
interpretation of the data can be made, one must know the upper bound of the critical 
region. The upper bound is determined by knowing the level of significance a, the 
degrees of freedom for factors v, or k-1, and the degrees of freedom for error v, or 
k(n-1). Once this information is known, the upper bound of the critical region can be 
obtained from a standard table of Critical Values of the F Distribution [F,,,; 2]. For 
Fos, 1,22, the value of the upper bound is 4.3. If the relationship f,,,,,,>f, is true, then the 


null hypothesis (Ho) should be rejected. In this example, 168.9>4.3: therefore, the null 


Table III.4 and subsequent Anova tabics were generated using the statistics software 
package Minitab. 
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hypothesis is rejected and the alternative is accepted. Interpretation of the data findings 
mean that there is a difference in population means as a result of CIP at a 95 percent 
confidence interval (CI). 

Once a change has been detected in a logistics parameter and it is known that 
CIP was the cause of the change, the task becomes one of quantifying the net benefits in 


terms of life cycle cost reductions and dollar savings.'* 


'° For methods to quantify benefits readers should consult a life cycle cost reference 
such as; Blanchard, B. S., Design and manage to Life Cycle Cost. Matrix Press, 1978; 
Blanchard, B. S., Logistics Engineering and Management, Prentice-hall, 1986; Life-Cycle 
Cost Model Final User’s Manual and Operating Instructions, Vol. I, Planning Research 
Corporation, PRC-R-1225, Arlington, VA; or Equipment Designer’s Cost Analysis 
System, Systems Exchange, 5504 Garth Avenue, Los Angles CA 90056. 
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IV. ANALYSIS 
In this Chapter, the results are explained which were obtained by using the model 


and technique: of this thesis in conjunction with ECIFR data. 


A. OBJECTIVES 
The objectives of the analysis are: 
* to determine if the effect of expending CIP resources was large enough to be 
detected by an observab'= change in component MTBF. 


* to demonstrate if an observed change in MTBF was in agreement with the model 
in Chapter I. 


* to determine if the change was statistically significant at the 95 percent confidence 
interval by using the ANOVA technique. 


* to show that there is a positive relationship between CIP and MTBF improvements. 


B. PRESENTATION OF DATA 

Figures 4.1, 4.2 and Table IV.1 are illustrations of data from an actual engine 
component which met the selection criteria discussed in Chapter II. This particular 
component helps demonstrate the complex nature of isolating the benefits of a single CIP 


action from other factors such as overlapping CIP efforts. 
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Figure 4.1 Aterbumer Igniter Fuel Valve Observed Monthly MTBF 
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Figure 4.2 Afterburner Igniter Fuel Valve Cumulative Failures Versus Cumulative Engine 
Hours. 
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The component represented by the above Figures is the TF30 Afterbumer Igniter 
Fuel Valve. The TF30 is produced by Pratt & Whitney and is the power plant for the F- 
14 Tomcat. The F-14 Tomcat was introduced into the fleet in the early 1970’s. The data 
set for the Afterburner Igniter Fuel Valve is without break, but because the TF30 became 
operational in the early 1970’s, the entire life of the cumponent’s MTBF is not 
represented. 

After talking with both Pratt & Whitney program engineers and Navy CIP engine 
managers, the authors learned that the Afterburner Igniter Fuel Valve was the recipient 
numerous CIP efforts during the period which the data covers (no fewer than three times 
but not more than ten times). In this situation, how does one distinguish or weight 
increases in MTBF among all of the ECPs which have taken place. 

By Figure 4.1, one can see that there has been a change in the MTBF of the 
' Afterbumer Igniter Fuel Valve from approximately 1,000 hours in 1982 to in excess of 
6,000 hours in 1989. Because there were multiple CIP efforts affecting the MTBF of the 
Afterbumer Igniter Fuel Valve, there was no abrupt change in the MTBF at any point in 
the graph. The plot of MTBF versus months indicates that there was a steady increase 
in the MTBF throughout the life of the component. The observations from Figure 4.1 are 
confirmed by Figure 4.2. Figure 4.2 also demonstrates that the component has had an 
increasing MTBF and a decreasing A throughout the data set. 

Since the Afterburner Igniter Fuel Valve has been re-engineered multiple times, one 


could surmise that there would not be any obvious demarkation of changes in MTBF. 





One would expect to see gradual changes in MTBF caused by the overlap of different 


ECP’s. This expectation is substantiated by Figures 4.1 and 4.2. 


Table IV. 1 Afterbumer Igniter Fuel Valve ANOVA 


WUC 23B78 

SOURCE ss MS F p 
FACTOR 1.296E+09 648198144 41.00 0.000 
ERROR 2.039E+09 15809329 

TOTAL 3.336E+09 


LEVEL MEAN 
MTBFa 1008 
MTBFb 2482 
MTBFc 8276 


POOLED STDEV = 3976 


INDIVIDUAL 95 PCT CI’S FOR MEAN 
BASED ON POOLED STDEV 





Using Figure 4.2, the data were divided into three sub-sets. The first sub-set 
included the data from January 1979 to October 1980 (300,600 EFH); the second 
contained data from November 1980 to February 1985 (1,143,826 EFH); and the third 
contained data from March 1985 to December 1989 (2,302,322 EFH). From Table IV.1, 
one could conclude that at the 95 percent confidence level there was not a statistically 
significant difference between sub-sets | and 2. There was, however, a statistically 


significant difference between sub-sets | and 3, and, sub-sets 2 and 3. 
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C. CONCLUSIONS 
In summary, one can make the following conclusions about CIP and the Afterbumer 
Igniter Fuel Valve: 
« the affect of CIP resources was detected and observed in Figure 4.1 by the 
continual increase in component MTBF from 1979 to 1989. 


¢ the trend in failure rate illustrated in Figure 4.2 is consistent with the model in 
Chapter Ill (see Figure 3.5). 


* there is a Statistically significant difference between 2 of 3 Afterbumer Igniter Fuel 
Valve data sub-sets (see Table IV.1). 


* the more CIP resources which were expended on the Afterbumer Igniter Fuel 
Valve, the more the failure rate decreased and MTBF increased (see Figures 4.1 and 
4.2). 

The analysis clearly shows that there is a positive relationship between MTBF 


improvement and CIP. The changes in MTBF for the Afterburner Igniter Fuel Valve are 


detected as they are predicted by the methodology and analytical procedures of this thesis. 








V. CONCLUSIONS AND RECOMMENDATIONS 


A. SUMMARY 

The original goal of this thesis was to provide managers of the Component 
Improvement Program with a tool that would quantify the benefits of CIP. One of the 
original assumptions underlying the research effort was that CIP funding needed a 
significant increase rather than an incremental increase. As the research effort continued, 
it became obvious to the authors that quantifying the benefits of CIP wac an extremely 
complex process. 

First, the budgeting process within DON was examined. The drop in CIP funds 
between 1983 and 1987 was the result of internal Navy management. An attempt was 
made to determine if this decline in budget was appropriate or if another method to justify 
CIP funding level requests was required. We believe a tool to help budget managers 
associate the costs and benefits of CIP is required. This tool should be designed to assist 
budget managers in determining an efficient allocation of resources for the Component 
Improvement Program. 

After discussions with NAVAIR and OPNAV personnel, it seemed logical to search 
for CIP benefits which accrued at the component level, rather than at the system level. 
A theoretical discussion was presented conceming why the effects of observable benefits 


became diluted as they filtered through the component, subsystem and system structure 
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of an engine. This discussion supports the intuition that the component level is the 
correct place to assess CIP benefits. 

The tools for selecting and analyzing a component’s MTBF were developed and 
validated in Chapters II, II and IV. As a result of developing this procedure, the 
problems of normalizing the ECIFR data against the CIP milestone timeline became 
evident. There exists some evidence that there is a database structuring problem; that is, 
data from the ECIFR are not collected in a format compatible with the previously 
mentioned timeline. 

Without the ability to normalize the ECIFR data, it is impossible to effectively 
determine the level of change in component MTBF which has occurred as a result of 
specific CIP activities. 

The most significant accomplishments made by the authors were: 

* analysis of the problems involved in developing a methodology for selecting 
candidate components for study. 

* use of the analysis of variance for one-way classification as a technique for 
discriminating between sample means of component MTBF; this technique is 
applicable to other logistics parameters. 

¢ development of formats for graphing ECIFR data in its existing form and 
explanation of the significance of these graphs. 

A deeper understanding of the Component Improvement Program process was 
achieved. By producing a procedure for analyzing ECIFR data and other logistics 
parameter data, this thesis has clearly paved the way for additional research. This thesis 


has provided a comprehensive bibliography which lists other publications about CIP and 
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continuing engineering. The bibliography eliminates the need to extensively research 


background and historical information. 


B. CONCLUSIONS 

The effects of CIP can only be evaluated at the component level. Because of the 
complex interactions of components and their intricate relationships, the effects of CIP 
are lost in the random noise of a system. The effects of a single CIP expenditure are so 
small when examined at the system level they become insignificant; therefore, engine 
parameters should not be tracked at the system level but at the component level. 

Sub-optimization of dollar resources is occurring. Because different individuals 
with different incentive structures are responsible for managing and executing the funds 
required to take an aircraft from "cradle to grave," the Navy is accepting a suboptimal 
expenditure of resources. Many tradeoffs take place when designing and operating an 
aircraft. A single individual (or organization) should be responsible for making those 
tradeoffs. The same individual should also be responsible for executing the budget and 
supporting the weapon system. One goal should be to achieve the most "effective and 
reliable" system within a given budgetary constraint which includes RDT&E, APN, and 
O&M,N dollars. Because of the way the United States Navy currently manages funds, 
each appropriation manager is optimizing the use of his funds largely without regard to 
the impact on the managers of other appropriations. If one individual or organization 
were responsible to manage an allocation of funds whose purpose was "cradle to grave" 
support of a weapon system, he/she could make the tradeoffs necessary to ensure the 


optimal use all funds and still meet the system’s mission objectives. 
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Currently OP-05 is responsible for programming and planning allocations between 
different appropriation resources; however, by the time field data reach the decision maker 
in OP-05 it is homogeneous. There is not enough detail to allow the resource manager 
to make a rational decision which achieves the most "effective and reliable" system within 
a given budgetary constraint. 

We conclude from this project that the method of data collection plays a critical role 
in developing the capability to measure the effects of CIP on logistics parameters and 
quantifying the benefits of CIP itself. Once the correct format of data is available, it 
should be possible to calculate the net benefits in dollar terms. 

The component we selected for analysis demonstrates the difficulties in measuring 
the incremental benefits from additional CIP expenditures. Our objective was to assess 
the consequences of a CIP expenditure as soon as possible after the installation of an 
’ ECP. However, if evaluation is based on the existing data, an accurate assessment is 
unlikely until a very large amount of program time has elapsed. 

Perhaps it is best to recognize that it is probably not possible to measure CIP 
productivity using active engine programs. Rather, CIP productivity might best be 
assessed by looking at "retired" engine programs. One could then assess the value or 
utility of continuing a component improvement program from a life-cycle-cost basis rather 


than on a year to year incremental! basis. 


C. RECOMMENDATIONS 
The following recommendations are provided as a means of further achieving the 


objectives stated in Chapter I. 
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The authority for "cradle to grave" management of an aircraft should be 
consolidated in one program management office. This action would allow optimal 
tradeoffs to be made and the impact of these tradeoffs to be evaluated across the entire 
spectrum of a weapon system life cycle. Tradeoffs could be made which consider 
impacts on reliability, maintainability, operability, operating and support cost, production 
and retro-fit cost, and weapon system effectiveness. 

The Aircraft Turbine Engine Component Improvement Program should be returned 
to the Aircraft Procurement, Navy (APN) appropriation. This initial step would overcome 
the sub-optimization problem. It would allow the appropriation manager to evaluate 
tradeoffs such as conducting a CIP effort and not providing the funds to implement the 
engine modifications. 

If CIP is to remain a separate RDT&E line item it should be split into three sub-line 
items. The first proposed line item would contain the funding needed to accomplish 
safety of flight objectives. The second proposed line item would contain the funding 
needed to achieve the remaining objectives of CIP. The final proposed line item would 
contain the funding needed to support CIP program management and infrastructure. This 
recommendation suggests a means to show how much money is being spent on safety of 
flight items and program management and how little is being spent on the remaining CIP 
objectives. 

A more indepth follow-on study should be conducted using the methodology and 
techniques discussed in Chapters I] and II]. Component selection criteria must be strictly 


followed. Once components are identified, NAVAIR engine program engineers should 
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determine the dates when the component reached each of the CIP timeline milestones. 
Disregard any component where T, and T, are not identified. 

Students who are in the Operations Research, Computer Systems Management, 
Financial Management, Material Logistics Support Management, and Aeronautical 
Engineering Curriculums should be recruited to study the Component Improvement 
Program further. Because of the multi-faceted nature of the Component Improvement 
Program, students in each of these curricula would be able to improve on this base of 
knowledge. A Computer Systems Management student could provide useful insight into 
a database design that would facilitate analysis using the methods of this thesis. A 
Material Logistic Support Management student could evaluate the effects of CIP on a life- 
cycle cost basis. A Financial Management student could develop a methodology for 
determining the efficient allocation of funds for the Aircraft Turbine Engine Component 
Improvement Program given current real world budgetary constraints. An Operations 
Research student could develop other analytical techniques to evaluate or determine 
optimal tradeoffs between recommendations which other disciplines might make. Finally, 
an Aeronautical Engineering Student might provide insight into the specific problems of 
aircraft engine design and support. The Aircraft Turbine Engine Component Improvement 


Program is a multi-disciplined program; therefore, the study of CIP budgeting should be 


open to the above disciplines. 
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D. AREAS FOR FURTHER RESEARCH 


Research should be conducted to determine if CIP is the causative factor in the 
improvement of MTBF. This thesis clearly demonstrates a positive relationship between 
the expenditure of CIP funds and improvements in MTBF. It remains to be shown that 
the expenditure of CIP resources was the cause of the improvement in MTBF. 

A cost savings analysis should be conducted on the J52 Power Plant Change (PPC) 
290. A claim is made by the J52 program engineer that if PPC 290 was incorporated, the 
O&M,N savings which might accrue from installation would also make it possible to 
procure a new radar system for the A-6. Instead, the decision appears to be to buy a new 
radar system and forego the O& M,N savings that could be achieved by incorporating PPC 
290. The apparent driver in this decision is the unavailability of funds to do both 
simultaneously. If the PPC were installed first, it might save enough money to pay for the 
radar system, too. 

The component sample size of this study should be expanded. With a sufficiently 
large sample size it may be possible to quantify the point where marginal benefits of CIP 
equal the marginal cost of CIP. In an economic sense, this is the proper allocation of 
funds for the Component Improvement Program. 

Alternative sources of data should be studied to apply to the model contained in this 
study. 

The effects of optimizing the use of funds at the appropriation level should be 
evaluated. These data should be compared this to what might happen if a fixed amount 


of funds had been used optimally over the life-cycle of the system. 
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The impact of "fully funding" the CIP program should be evaluated, i-e., zero 
funding shortfalls. Is the program executable? Can the contractors support the program? 
Do the contractors have on staff or access to the technical and management expertise to 


support a "fully funded" CIP program. 





This Appendix contains the data used to generate the graphs and ANOVA tables in 


chapter 4. 


RAW DATA FROM ECIFR 


DATE FAIL 


7901 
7902 
7903 
7904 
7905 
7906 
7907 
7TO90€e 
7909 
7910 
7911 
7912 
8001 
8002 
8003 
8004 
8005 
8006 
8007 
8008 
8009 
8010 
8011 
8012 
8101 
8102 
8103 
8104 
8105 
8106 
8107 
8108 
8109 
8110 





EFH 


20572 
17100 
22584 
22176 
18536 
10476 

9730 
10580 

9564 
12154 
10434 

9782 

9648 

9852 
13528 
12972 
14786 
14736 
13628 
12688 
13774 
11300 
13878 
10558 
12888 
13366 
10504 
12536 
14436 
17830 
16090 
15576 
17952 
15000 


APPENDIX 


MTBF 


WUC23B78 
1033.6 
855.0 
1411.5 
852.9 
487.8 
1746.0 
608.1 
587.8 
478.2 
434.1 
869.5 
489.1 
742.2 
895.6 
1503.1 
1853.1 
1232.2 
1133.5 
1238.9 
1057.3 
1530.4 
1130.0 
1542.0 
1319.8 
920.6 
1909.4 
583.6 
1044.7 
2062.3 
1981.1 
1340.8 
1112.6 
1196.8 
1000.0 
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COMPUTED DATA 


CUMEFH CUMFAIL 


20572 

37672 

60256 

82432 
100968 
111444 
121174 
131754 
141318 
153472 
163906 
173688 
183336 
193188 
206716 
219688 
234474 
249210 
262838 
275526 
289300 
300600 
314478 
325036 
337924 
351290 
361794 
374330 
388766 
406596 
422686 
438262 
456214 
471214 














RAW DATA FROM ECIFR 


DATE 


6111 
8112 
8201 
8202 
8203 
8204 
8205 
8206 
8207 
8208 
8209 
8210 
8211 
8212 
8301 
8302 
8303 
8304 
8305 
8306 
8307 
8308 
8309 
8310 
8311 
8312 
8401 
8402 
8403 
8404 
8405 
8406 
8407 
8408 
8409 
8410 
8411 
8412 
8501 
8502 
8503 
8504 
8505 
8506 
8507 
8508 
8509 
8510 


FAIL 


ra 
ONKDFBDwDHOHDIOW 


e 


WUNWWDWOTDODWHEOHAN UO 


EFH 


15336 
10530 
15118 
14504 
16396 
17054 
14532 
16914 
28088 
14552 
15032 
19356 
14120 
13648 
15272 
13946 
20296 
18572 
17566 
15028 
13700 
20478 
17452 
17780 
18378 
14752 
17866 
18406 
18410 
16138 
18640 
19628 
19138 
22440 
18058 
18900 
18028 
14568 
19074 
14918 
19796 
18086 
18292 
18814 
20826 
21230 
20504 
18534 


MTBF 


1179.7 
752.1 
1079.9 
1036.0 
1490.5 
1003.2 
908.3 
1301.1 
2593.5 
1039.4 
2147.4 
1209.8 
1283.6 
1240.7 
2181.7 
1394.6 
1353.1 
2321.5 
1171.1 
1502.8 
4566.7 
2275.3 
2493.1 
2963.3 
1837.8 
2458.7 
2233.3 
4601.5 
4602.5 
2689.7 
1864.0 
2453.5 
3827.6 
11220.0 
4514.5 
3150.0 
18028.0 
4856.0 
2384.3 
1864.8 
2828.0 
2009.6 
2286.5 
6271.3 
6942.0 
10615.0 
4100.8 
6178.0 
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CUMEFH 


486550 
497080 
512198 
526702 
543098 
560152 
574684 
591598 
619686 
634238 
649270 
668626 
682746 
696394 
711666 
725612 
745908 
764480 
782046 
797074 
810774 
831252 
848704 
866484 
884862 
899614 
917480 
935886 
954296 
970434 
989074 
1008702 
1027840 
1050280 
1068338 
1087238 
1105266 
1119834 
1138908 
1153826 
1173622 
1191708 
1210000 
1228814 
1249640 
1270870 
1291374 
1309908 


COMPUTED DATA 


CUMFAIL 





RAW DATA FROM ECIFR 


DATE FAIL 


8511 
8512 
8601 
8602 
8603 
8604 
8605 
8606 
8607 
8608 
8609 
8610 
8611 
8612 
8701 
8702 
8703 
8704 
8705 
8706 
8707 
8708 
8709 
8710 
8711 
8712 
8801 
8802 
8803 
8804 
8805 
8806 
8807 
8808 
8809 
8810 
8811 
8812 
8901 
8902 
8903 
8904 
8905 
8906 
8907 
8908 
8909 
8910 


WAUBWERNHNNBDBWWHUWONNOWHAEFPNBHENWHRENURPUANE WWON BWWW BOON W 


EFH 


18320 
15100 
20154 
19186 
23130 
22310 
20292 
19916 
19928 
19878 
19412 
20854 
20096 
16058 
18124 
18836 
20378 
20276 
21016 
21362 
19172 
21226 
21488 
20936 
18434 
15088 
19154 
20216 
21590 
21002 
22180 
22152 
19594 
20916 
20924 
18692 
19322 
17856 
20508 
20082 
20718 
21510 
21932 
20464 
19468 
21384 
19022 
23394 


MTBF 


6106.7 
71550:..0 
2239.3 
3837.2 
5782.5 
7436.7 
6764.0 
6638.7 
4982.0 
9939.0 
21000.0 
6951.3 
2870.9 
16058.0 
9062.0 
3139.3 
4075.6 
20276.0 
4203.2 
10681.0 
19172.0 
5306.5 
7162.7 
10468.0 
18434.0 
3772.0 
9577.6 
20216.0 
3598...3 
7000.7 
21000.0 
11076.0 
9797.0 
21000.0 
6974.7 
3738.4 
19322.0 
5952.0 
6836.0 
5020.5 
10359.0 
10755.0 
5483.0 
6821.3 
4867.0 
4276.8 
3170.3 
AVIS 0 
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COMPUTED DATA 





CUMEFH CUMFAIL 


1328228 
1343328 
1363482 
1382668 
1405798 
1428108 
1448400 
1468316 
1488244 
1508122 
1527534 
1548388 
1568484 
1584542 
1602666 
1621502 
1641880 
1662156 
1683172 
1704534 
1723706 
1744932 
1766420 
1787356 
1805790 
1820878 
1840032 
1860248 
1881838 
1902840 
1925020 
1947172 
1966766 
1987682 
2008606 
2027298 
2046620 
2064476 
2084984 
2105066 
2125784 
2147294 
2169226 
2189690 
2209158 
2230542 
2249564 
2272958 





895 
897 
906 
911 








RAW DATA FROM ECIFR COMPUTED DATA 


DATE FAIL EFH MTBF CUMEFH CUMFAIL 
8911 2 15614 7807.0 2288572 1043 
8912 mo 13750 4583.3 2302322 1046 


os? 
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